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ABSTRACT 

A total of 50 college students enrolled in different 
sections of a language arts methodology course served as treatment 
and control groups to investigate the effects of student-generated 
questions on test performance. Periodically throughout the course, 
students in the treatment group received instruction in effective 
questioning techniques by discussing and evaluating hypothetical test 
questions produced by the professor and participating in a reciprocal 
questioning strategy, known as ''ReQuest,*' designed to help students 
think as they read. Treatment group students were also required to 
compose two multiple-choice and two true-false questions for the 12 
chapters in the course textbook. Three tests and a comprehensive 
final examination served as the independent measures. The results of 
analysis of covariance and the Johnson-Neyman Technique indicate that 
the treatment condition was effective only for students with higher 
levels of prior knowledge related to language arts methodology upon 
entry into the course, and that it was detrimental to students who 
had lower levels of prior knowledge, as reflected in the pretest 
scores. (TJH) 
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Abstract 

Col 1 ege students enrol 1 ed in di f f erent secrt i ons of a 
language arts methodology course served as treatment and control 
groups to investigate the effects of student-generated questions 
on test performance. Three tests and a comprehensive final exam 
served as the dependent measures. Results of ANCDVA and the 
Johnson-Neyman Technique indicated that the treatment condition 
was effective (p ; .05) only for students with higher levels of 
prior knowledge related to language arts methodology upon entry 
into the course, and it was detrimental to students who had 
lower levels of prior knowledge, as reflected by pretest scores. 
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EFFECTS OF STUDENT-GENERATED QUESTIONS 
ON TEST PERFORMANCE 

Introducti on 

The value of using adjunct questions to enhance student 
comprehension of text and to encourage the development of 
thinking skills has been reported (Durkin, 1981; Graves fy Clark, 
1981; Redfield S< Rousseau, 1981; Reynolds Zi Anderson, 1980; 
Tierney ?/ Cunningham, 1980). However, several variables related 
to questioning need further study, such as: the amount and type 
of questioning, the identification of learning situations in 
which questioning is of most benefit, and the significance of the 
source (i.e., text, teacher, or student) of questioning. 

There is some evidence that students gain more from tey.t 
when they generate their own questions as opposed to responding 
to teacher-produced questions (Anderson ?/ Armbruster, 1980; Andre 

Anderson, 1978-79). Devine (1986) believes that having 
students prepare an actual test after reading can help them 
determine main points, relevant information, and the writer's 
purpose. But, other researchers (Herber Nelson, 1975; 
Rumelhart, 1980, 1984) are concerned that students usually lack 
effective questioning skills and may not have acquired sufficient 
schemata for the topic/concept being studied to permit higher 
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levels of questioning to occur. 

It may be that student-generated questions which are too 
general in nature, as well as questions which emphasize 
insignifi cant detai 1 s , may be detri mental to the growth o-f 
thinking skills. The purpose of this study was to examine the 
effects of student-generated questions on the test performance of 
col 1 eg e students maj or i ng in el ementary educati on . 

Methodol ogy 

Subjects . A total of 50 students, who were classified as 
juniors, were involved in the study which lasted for one 
semester. The treatment group contained 26 students, and the 
control grouD contained 24 students. No significant 
differences were found between the two groups in either their 
mean ACT scores (t(46) = 0.82, ns) or their mean GPAs (t(46) = 
1.36, ns) . GPAs and ACT scores were not available for two of 
the students. Mean ACT and GPA scores for the treatment and 
control groups were 15.72 (ACT^) , 2.90 (GPA^) , 16.48 (ACTc=), 
and 3.15 (GPAc=). The two groups of students were enrolled in 
different sections of a language arts methods course, both of 
which were taught by the first author. 

Procedures . At the beginning of the course, both classes 
were given a 25-item pretest, containing 15 mul tiple-choice and 
10 true-f al se i tems , cover i ng maj or language arts concepts . 
The pretest items assessed only the knowledge category of 
Bloom's (1956) taxonomy. During the course, both groups were 
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given three tests (each covering four chapters) and a 
comprehensive final exam. The same tests were used with both 
groups, and all tests contained multiple-choice and true-false 
items composed by the professor. Each of the four tests 
contained items representing each of the six major categories 
(i.e. , knowledge, comprehension, application, analysis, 
synthesis, and evaluation) identified by Bloom (1956). 

Periodically throughout the course, the treatment group 
received instruction in effective questioning techniques by (1) 
discussing and evaluating hypothetical test questions produced 
by the professor, and ^2) Darticipating in a reciprocal 
questioning strategy, known as ReQuest (Manzo, 1969), which is 
designed to help students think as they read. ReQuest 
encourages students to generate their own questions about the 
content material under study. The strategy requires both 
teacher and students to take turns asking each other questions 
about a portion of text material they have read. During the 
procedure, the teacher's role as a good questioner serves as a 
model that students begin to emulate in the formation of their 
questions. Also, the students were required to compose four 
test questions (2 multiple-choice and 2 true-false) for each of 
12 chapters in the text, after the chapter had been discussed 
in class. The test questions were collected every two weeks by 
the professor who encouraged the students to write questions 
requiring more than literal recall. As an incentive to get 
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students to think about the te;:t material, the professor told 
the students that those questions which represented levels of 
cognitive processing beyond the knowledge level might oe 
considered for inclusion on the final exam. However, none of 
the tests contained specific items written by the students. 

The students in the control group were encouraged to ask 
questions during class discussions about the content of each 
chapter of the text. During lectures the professor posed 
questions, representing the si;: major categories of Bl oom's 
(1956) taxonomy, to facilitate discussion of major concepts 
contained in each chapter and to check students' understanding 
of the text material. 

Data Analysis . The three tests admi ni stered dur i ng the 
course and the final exam served as the dependent variables, 
rB^BrrBd to as Tests 1, 2, 3, and 4, respectively. Failure to 
meet the assumption of homogeneity of regression precluded the 
use of ANCOVA for three of the four dependent measures. For 
these three measures, (i.e.. Tests 1, 3, and 4) the Johnson- 
Neyman Technique (see Kerlinger ?/ Pedhazur, 1973) was used to 
determine pretest score regions in which the two groups 
di f f ered. 

Resul ts 

Tests of the assumotion of homogeneity of regression 
resulted in the rejection of equal regression slopes for Test 1 
(p < .04), Test 3 (p < .01), and Test 4 (p < .03). Homogeneity 
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of regression was accepted only for Test 2, Correlations 
between the pretest scores and scores on each of the four tests 
are presented in Table 1. The pretest scores were not found to 
be significantly related to scores on any of the four dependent 
variables for the control group; however, the pretest scores 
were signficantly and positively related to scores on each of 
the four variables for the treatment group. 



Insert Table 1 about here 



Analysis of cova-iance for the Test 2 scores, using the 
pretest as the covariate, indicated no significant differences 
in the adjusted means for the treatment and control groups. 
The adjusted means for the treatment and control groups were 
79.22 and 75.69, respectively. 

Since scores on Tests 1, 3, and 4 did not meet the 
assumption of homogeneity of regression, the Johnson-Neyman 
technique was used to analyze scores on these tests. The 
Johnson-Neyman technique defines regions along a predictor 
variable dimension where significant differences e):ist between 
two regression lines. However, prior to this analysis, the 
pretest score means were compared using a t-test. No 
significant pretest score differences were found between the 
two groups (t(48) = 0.18, ns) . 

Application of the Johnson-Neyman technique to the 
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treatment and control group regression lines for the Test 1 
scores indicated that the performance of the treatment and 
control groups differed significantly only for those students 
scoring '^elow 46.87. on the pretest. Examination of the plots 
of the pretest-Test 1 regression lines for the two groups 
(Figure 1), indicates higher performance on Test 1 for students 
i n the control group . In other words, student-generated 
questions had a detrimental effect on Test i scores for those 
students scoring below 46.87. on the pretest with no differences 
in performance for those scoring above 46.8%. 



Insert Figure 1 about here 



Analysis of the Test 3 scores using the Johnson-Neyman 
technique resulted in two regions of significance. The 
treatment and control group students were found to differ in 
their Test 3 performance when their pretest scores were less 
than 42. 6X and when their pretest performance was greater than 
58.4/1. Examination of the pretest-Test 3 regression lines for 
the two groups indicates that when pretest performance was 
greater than 58. 4X, the treatment group outperformed the 
control group on Test 3; however, when pretest performance was 
less than 42.6%, the control group outperformed the treatment 
group . 

The results of application of the Johnson-Neyman technique 
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to the Test 4 scores yielded results similar to those found 
with Test 3. The regions o-f significance were -for pretest 
scores below 42.3% and for pretest scores above 70.8'/.. When 
pretest performance exceeded 70.8%, the treatrrient students 
outperformed the control group students, however, when pretest 
performance was less than 42.3%, the control students 
outperformed the treatment group students. 

Di scussi on 

In general , for the control group there was no 
relationship between pretest performance and performance on 
each of the dependent variables. However, for the treatment 
group there were significant, positive relationships between 
the pretest and each of the dependent variables. The pattern 
of performance on each of the four dependent variables was 
basically the same: treatment-group students scoring high on 
the pretest outperformed corresponding students in the control 
group, and conversely, control -group studants scoring low on 
the pretest outperformed corresponding students in the 
treatment group. 

Do student-generated questions improve test scores of 
students? The general finding was that the treatment condition 
had a positive effect on students with higher levels of prior 
knowledge related to language arts methodology upon entry into 
the course and a negative effect on students with lower levels of 
prior knowler'ge. The finding that student-generated questions 
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had a detrimental effect on test performance of students with 
lower levels of prior knowledge was surprising, although it 
offers some support for the concerns expres-^ed by other 
researchers (Herber and Nelson, 1975; Rumelhart, 1980, 1984) 
related to the need for sufficient schemata as a prerequisite for 
effective questioning. This finding conflicts somewhat with that 
of Andre and Anderson (1978-79) who found that student generation 
of questions during study was more effective for lower than for 
higher verbal ability students. Although verbal ability and 
pri or e^per i ence are not <r /nonomous , both are related to 
comprehension performance. Therefore, one wonders if the 
performance of the students in the Andre and Anderson (1978-79) 
experiments would have remained stable had the study involved 
more than one testing session. 

Based on the findings of Brown and Smiley (1978), the 
investigators of this study had anticipated that requiring 
students to generate questions would encourage them to spend 
more time in direct interaction with text material, hence 
improving their test performance. Although students in the 
treatment group stated that composing the required questions 
too^ a lot of time, it doesn't appear that the increase in 
study time was of equal benefit to students with higher and 
lower levels of prior knowledge related to the content Df the 
course. 

One could argue that the results of Tests 3 and 4 indicate 
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that higher levels of prior knowledge combined with the 
cumul ati ve effects of student -generated quest i ons have 
significant, positive affects on test performance. However, 
student-generated questions in combination with higher levels of 
prior knowledge do not appear to produce immediate positive 
effects. 

The results of this study support the general conclusion 
that student-generated questions may not benefit students who 
are just beginning to develop schemata related to language arts 
methodology, but for those students whose schemata is better 
developed in the area of language arts the process may be quite 
beneficial. This conclusion is in agreement with ideas related 
to prior knowledge which have been e;:pressed by Bransford 
(1979) and other cognitive psychologists (cf Ausubel , 1963, 
1968; Buhler, 1908). In the words of Bransford (1979), "simply 
having prior knowledge when making comprehension ratings or 
recalling is not necessarily sufficient to ensure adequate 
results. Previous knowledge must be activated in order to 
facilitate one's current abilities to understand and learn" (p. 
135). Whether or not the learner's prior ^nowledge becomes 
activated seems to depend on one or more of the following 
variables: the extent of prior knowledge as well as the 
conditions under which the knowledge was acquired; the 
sequential nature of learning; the role of practice in the 
acquisition and mastery of new concepts and s^:ills; the 
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conditions that -facilitate transfer o-f knowledge to new 
examples; and the degree to which certain types o-f experiences 
help students learn to learn. 

It IS possible th^t students in the treatment group who 
had lower levels o-f prior knowledge upon entry into the course 
needed to acquire a better understanding o-f basic language arts 
concepts prior to generating questions. It may be that 
students with lower levels o-f prior knowledge should not be 
required to generate questions about text material until they 
reach a speci-fied level o-f pro-ficiency in the categories o-f 
knowledge and comprehension. Requiring these students to 
compose questions about the text material may have caused them 
to concentrate on details rather than main ideas and practical 
application oi concepts. Also, it could be that the time these 
students spent composing questions could have been better used 
by them in practice activities related to the acquisition and 
mastery o-f new concepts. 

While prior knowledge seems to facilitate learning in 
general, the findings of this study suggest that future studies 
should explore the degree to which certain types of learning 
activities help students learn to learn and the conditions that 
facilitate transfer of knowledge to new examples. Also, further 
research is needed to (a) identify the specific types of learning 
experiences which complement varying levels of prior knowledge, 
and (b) determine whether or not student-^gener ated questions 
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have a cumulative e-f-fect on tesc performance. 
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Table 1 



Correl ati ons 


Between Pretest 


Scores and Scores on 


Tests 1-4 


For the Treatment 


and Control Groups 


Test 


Control 


Treatment 


1 


. 003 


.537** 


2 


. 129 


.519** 


3 


-. 227 


.441* 


4 


. 019 


.541** 



• p < .U5 
••p < .01 
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Figure 1. Regions of nonsignif icance for each variable. 
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